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Abstract 

This  research  investigates  the  possibility  of  prioritizing  and  quantifying  of  C-17 
pilot  training/currency  flight  events.  Currently,  the  1 1-2C-17  Volume  1  does  not 
prioritize  training  events,  only  mandates  the  minimum  number  of  flight  events  a  pilot 
must  complete  within  a  given  period  of  time.  Through  interviews  and  surveys  of  C-17 
experts,  including  high-time  instructor  and  evaluator  pilots,  this  research  provides 
quantifiable  coefficients  for  each  of  twenty-eight  selected  C-17  flight  events.  The 
coefficients  are  calculated  as  a  product  of  the  impact  rankings  of  four  categories  labeled 
as  an  SPR  Score:  Skill  Required,  Skill  Diminish  Rate,  Probability  of  Occurrence,  and 
Risk.  This  SPR  Score  provides  decision  makers  from  the  squadron  level  up  through  and 
including  MAJCOM  level  a  means  by  which  to  prioritize  training  events  within  a  limited 
resource  environment.  Additionally,  this  coefficient  allows  the  future  possibility  of 
optimization  of  C-17  sorties  by  linear  programming  to  ensure  AMC  that  each  flight  hour 
and  pound  of  jet  fuel  is  being  maximized  to  the  fullest  utilization  of  providing  the 
combatant  commanders  with  the  most  proficient  and  trained  C-17  pilots  possible. 
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I.  Introduction 


General  Issue 

U.S.  Air  Force  C-17  pilots  are  expected  to  be  mission  effective  under  the  harshest 
and  most  demanding  conditions.  The  flexible  and  responsive  nature  of  the  Air  Force’s 
core  mission  necessitates  a  high  state  of  readiness  for  its  pilot  force.  Currently,  C-17  pilot 
readiness  is  determined  almost  exclusively  via  currency  status.  That  is,  a  pilot  is 
considered  “ready  to  fight”  if  they  have  accomplished  all  required  events  within  a  certain 
maximum  allowed  time  period  of  the  date  on  which  they  plan  to  fly.  The  maximum  time 
periods  allowed  per  event  are  regulated  by  the  AFI  1 1-2C-17  volume  1  for  all  C-17 
events. 

Problem  Statement 

C-17  Aircrew  Training  Program  (ATP)  is  designed  to  provide  a  unit  with  the 
procedure  to  ensure  its  pilots  maintain  a  minimum  acceptable  level  of  proficiency.  The 
ATP  has  also  become  the  primary  method  of  determining  a  pilot’s  readiness  via  currency 
status.  It  is  not  designed  in  a  manner  in  which  unit  leadership  has  an  established  method 
to  ensure  they  are  providing  the  combatant  commander  with  the  most  qualified  and 
proficient  pilot  force,  only  the  most  current.  The  C-17  ATP  establishes  a  minimum 
amount  of  flight  events  that  will  maintain  a  pilot’s  status  as  “current”  and  considers  all 
flight  events  to  be  equally  important  as  it  pertains  to  currency  status.  A  program  does  not 
currently  exist  that  provides  unit  decision-makers  a  mathematical  method  to  ensure  the 
most  qualified  warfighters  are  being  sent  to  the  fight. 


Currency  status  is  intended  to  be  used  as  an  aid  to  decision  makers  in  the  unit  to 
help  identify  those  pilots  that  are  the  most  prepared  to  aviate  under  virtually  any  wartime 
or  peacetime  conditions.  Many  times,  a  gap  exists  between  currency  and  proficiency  and 
unit  leadership  is  provided  with  near  non-existent  authority  to  acknowledge  and  discard 
currency  status  in  order  to  place  a  more  qualified  pilot  on  a  crew.  Additionally,  large 
amounts  of  flight  hours  are  consumed  in  order  for  pilots  to  strive  for  and  obtain  currency 
in  flight  events  that  are  often  times  not  necessary  to  meet  mission  accomplishment  or 
contribute  to  overall  proficiency. 

Research  Objectives 

The  intent  of  this  research  is  to  explore  quantifiable  analytic  options  regarding  C- 
17  flight  training  in  order  to  provide  the  C-17  ATP  and  unit  leadership  with  a 
mathematical  model  or  system  that  can  be  to  ensure  the  most  qualified  flyers  are 
available  to  the  combatant  commander.  Any  results  implemented  could  potentially  lead 
to  more  effective  and  efficient  flight  hours,  more  proficient  pilots,  and  a  more  flexible 
and  adaptable  training  plan  for  flying  squadrons. 


Research  Focus 

The  focus  of  this  research  will  be  limited  to  C-17  readiness  and  currency 
requirements  defined  by  and  compared  to  AFI  1 1-2C-17  volume  1  conditions.  If  any 
changes  are  recommended,  a  C-17A  pilot  readiness  pilot  program  will  be  suggested  at  a 
sample  unit  in  order  to  test  and  compare  the  suggested  readiness  model  with  the  current 


ATP. 


Investigative  Questions 


1 .  What  is  the  current  AMC  C-17  training  model? 

2.  How  does  it  compare  to  the  rest  of  the  Air  Force  MDSs? 

3.  How  do  USAF  currency  requirements  differ  from  the  civilian  U.S. 
Airlines? 

4.  Are  all  C-17  flight  events  equal? 

5.  Can  C-17  training  events  be  quantified  and  stratified? 

6.  What  flexibility  is  gained  at  the  squadron,  group,  and  MAJCOM  level? 

7.  Feasibility?  (Hammer,  M.  and  Champy,  J.,  2001) 

a.  Technical  -  Can  it  be  done? 

b.  Economic  -  Can  we  afford  it? 

c.  Cultural  -  Will  our  people  go  for  it? 

Assumptions/Limitations 

1 .  All  maneuvering  events  logged  in  flight  or  in  the  simulator  (sim)  on  any  type 
of  training  or  operational  mission  are  considered  “flight  events”. 

2.  AFI  1 1-2MDS  volume  1  data  is  valid  through  submission  date  of  GRP. 

3.  Data  analysis  is  limited  to  Aircraft  Commander  Flight  Level  B. 

4.  Research  and  discussion  is  limited  to  continuation  training. 

5.  Statistical  data  collected  from  expert  sources  are  indicative  of  the  population 
when  large  sample  sizes  are  not  available. 


II.  Literature  Review 


Chapter  Overview 

The  1 1-2C-17  volume  l(Vol  1)  is  the  regulatory  guidance  written  by  Air  Mobility 
Command  (AMC)  for  all  C-17  training.  The  Vol  1  provides  aircrew  with  the  required 
minimum  amount  of  training  that  AMC  deems  a  pilot  will  need  to  operate  a  C-17  safely 
under  wartime  and  peacetime  conditions.  Pilots  may  complete  additional  training  in  order 
to  increase  proficiency  and/or  capability  in  a  given  flight  event.  Over  the  past  few  fiscal 
periods,  AMC  has  reduced  flight  events  or  moved  the  requirement  from  the  aircraft  to  the 
simulator  in  an  effort  to  seek  cost  savings.  Additionally,  further  reduction  of  flight  events 
is  under  investigation  due  to  their  perceived  necessity  or  likelihood  of  occurrence.  This 
change  has  resulted  in  less  flight  events  being  accomplished  in  the  aircraft  for  C-17 
pilots.  In  short,  financial  restrictions  are  reducing  the  capability  and  proficiency  of  the  C- 
17  pilot  force. 

MAJCOM  Approaches  to  Flight  Currency 

Before  deeply  investigating  the  C-17  pilot  training  model,  an  overview  of  Air 
Force-wide  approaches  to  flight  currency  should  be  discussed.  For  this  research,  two 
main  categories  will  be  used:  Mobility  Air  Forces  (MAF)  and  Combat  Air  Forces  (CAF). 
The  MAF  comprises  of  all  heavy  lift  and  air  refueling  airframes,  while  the  CAF 
encompasses  all  fighter  and  bomber  aircraft,  as  well  as  some  intelligence,  surveillance, 
and  reconnaissance  aircraft,  such  as  the  E-3.  Generally  speaking,  AMC  is  the  regulatory 
authority  for  all  MAF  aircraft,  while,  also  generally  speaking,  Air  Combat  Command  is 


the  authority  for  all  CAF  aircraft.  This  distinction  is  important  when  comparing  and 
contrasting  the  two  different  training  model  approaches  to  pilot  currency  and  proficiency. 

Each  airframe  (aircraft  type)  possesses  its  own  1 1-2MDS  volume  1  which  is  the 
regulation  and  instruction  authority  for  aircrew  continuation  and  recurrency  training. 
Aside  from  the  geographical  MAJCOMs  (PACAF,  USAFE,  etc.),  AMC  authors  the  Vol  1 
for  the  MAF  and  ACC  authors  the  Vol  1  for  the  CAF.  AMC’s  training  program  is  based 
almost  solely  on  currency  status  by  flight  event.  AMC  ensures  that  for  a  pilot  to  remain 
current,  he  must  accomplish  a  flight  event  within  a  maximum  period  of  time  since  last 
accomplishing  the  same  event.  Nearly  every  type  of  flight  event  is  accounted  for  in  the 
currency  tables  to  ensure  that  an  AMC  pilot  has  is  at  least  minimally  proficient  in 
virtually  any  conceivable  scenario.  The  CAF  model  is  significantly  different.  In  fighter 
units,  for  example,  pilots  must  remain  “current”  in  only  a  minimal  amount  of  “big  ticket” 
flight  events.  These  events  include  items  such  as  Night  Landing,  Air  Refueling,  and 
Instrument  Approach.  In  order  for  the  CAF  model  to  ensure  its  pilots  are  ready  to  fight,  it 
also  places  a  minimum  number  of  flights  that  a  pilot  must  accomplish  within  a  certain 
period  of  time  to  consider  that  pilot  “Mission  Ready”.  In  general,  the  MAJCOM  does  not 
dictate,  or  care,  what  those  pilots  do  on  those  minimum  number  of  flights,  so  long  as 
those  minimum  number  of  currency  items  are  accomplished  and  the  total  number  of 
sorties  meets  the  minimum  to  be  Mission  Ready.  It  is  almost  exclusively  left  to  the 
authority  of  the  squadron  or  group  to  decide  what  is  accomplished  on  that  minimum 
number  of  flights  so  that  the  pilots’  training  is  satisfactory  in  line  with  the  wing 
commander’s  vision,  as  he  is  ultimately  accountable  to  the  Numbered  Air  Force  (NAF) 
and  MAJCOM  for  the  readiness  of  his  pilots. 


The  distinction  between  the  MAF  and  the  CAF  is  significant  for  two  reasons. 

First,  it  displays  that  there  is  more  than  one  acceptable  method  to  ensure  USAF  pilots  are 
ready  to  deploy  and  fly  in  accordance  with  a  combatant  commander’s  (COCOM’s)  intent. 
Secondly,  it  shows  that  number  of  aircraft  currency/flight  events  is  not  a  direct 
correlation  to  pilot  proficiency.  The  following  figures  display  the  discrepancies  in 
currency  flight  events  among  USAF  airframes.  (Source:  1 1-2MDS  Vol  1) 
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Figure  2.1  Total  Number  of  Flight  Events  by  Mission  Type  (AFI 11-2MDS  Volume  1,  2006-2012) 

Figure  2.1  is  an  illustration  of  the  differences  in  approaches  to  flight  event 
currency  as  discussed  above.  AMC  has  adopted  a  similar  model  to  continuation  training 
as  that  used  in  civilian  aviation,  in  particular,  the  airline  industry.  This  is  not  surprising 
due  to  the  similar  characteristics  in  aircraft  between  AMC  and  the  airlines.  In  some  cases, 
such  as  the  KC-135  and  KC-10,  the  USAF  aircraft  is  a  modified  civilian  airliner. 
However,  the  question  begs  the  question,  “What  is  it  about  MAF  aircraft  that  require  five 
times  as  many  currency  flight  events  to  produce  a  minimally  proficient  pilot?” 


A  natural  skepticism  to  the  comparison  in  these  figures  flows  from  even  the  basic 
understanding  of  aircraft  mission  sets.  It  is  reasonable  to  question  the  validity  of  this 
comparison  due  to  the  markedly  different  missions  that  each  of  these  aircraft  possess. 
Superficially,  it  does  not  seem  reasonable  to  draw  conclusions  when  comparing  a  C-17 
heavy  airlift  aircraft  with  an  F-22  fifth  generation  stealth  fighter  aircraft.  Figure  2.2 
addresses  some  of  the  aircraft  comparison  problems.  The  data  in  Figure  2.2  contains  only 
strategic  flight  events.  Strategic  flight  events  are  only  those  events  that  any  generic 
aircraft  would  need  to  maintain  currency  in  order  to  get  “from  Point  A  to  Point  B”.  No 
flight  events  that  are  associated  with  missions  such  as  airdrop,  weapons  engagement,  etc. 
are  considered.  This  comparison  will  serve  as  a  more  useful  “apples  to  apples” 
comparison. 


Figure  2.2  “Strategic”  Flight  Events  by  MDS  (AFI 11-2MDS  Volume  1,  2006-2012) 

It  is  clear  from  Figure  2  that  AMC  continues  to  dominate  the  number  of  required 
flight  currency  events,  even  when  analyzed  at  the  MDS/airframe  level.  It  is  a  further 


possible  conclusion  that  number  of  currency  items  may  simply  be  correlated  to  size  of 
aircraft;  that  is  to  say,  the  larger  the  aircraft,  the  more  currency  items  are  necessary.  To 
further  solidify  the  comparison,  Figure  2.3  identifies  which  aircraft  are  large  frame 
aircraft.  While  all  6  aircraft  types  in  the  AMC  section  are  large  airframe,  six  of  the 
sample  large  frame  aircraft  also  belong  to  ACC’s  section  of  the  graph.  Therefore,  it 
cannot  be  deduced  that  a  large  frame  aircraft  necessitates  more  currency  flight  events. 
Furthennore,  Figure  2.4  highlights  that  identical  basic  airframes  exist  across  the 
MAJCOM  lines.  In  this  case,  the  Boeing  707  airframe  is  the  basic  structure  for  the  E-3, 
RC-135,  and  KC-135.  It  can  be  seen  that  ACC  deems  an  E-3  pilot  to  be  minimally  safe 
with  less  than  10  currency  items  while  it  takes  a  KC-135  pilot  in  AMC  more  than  60 
items  to  be  minimally  safe. 

Aside  from  simply  a  numbers  comparison,  the  impact  on  the  combatant 
commander,  our  ultimate  customer,  must  be  investigated.  For  a  typical  ACC  combat 
mission,  the  squadron  has  the  flexibility  to  tailor  virtually  all  of  their  training  sorties  to 
mimic  the  expected  combat  missions.  For  example,  if  an  F-22  squadron  is  expected  to 
deploy  in  order  to  provide  the  air  component  commander  with  air-to-ground  strike 
capability,  then  that  F-22  unit  will  spend  the  weeks  leading  up  to  the  combat  mission 
practicing  air-to-ground  strikes.  The  pilots  may  still  accomplish  their  continuation 
training  currency  items,  but  because  there  are  so  few  of  them  required,  it  does  not  hamper 
their  aircraft  time  available  to  train  for  the  expected  combat  role. 

In  contrast,  an  AMC  squadron  has  less  flexibility  due  to  regulatory  guidance.  If  a 
C-17  pilot  is  expected  to  be  tasked  with  a  combat  mission  or  deployment  to  the  Middle 
Eastern  theater,  only  marginal  ability  exists  to  tailor  training.  The  C-17  pilot  may  already 


be  aware  that  the  vast  majority  of  approaches  to  landings  expected  on  the  deployment 
will  be  tactical  approaches  to  visual  or  assault  landings.  However,  the  same  pilot,  while 
desiring  to  prepare  for  the  actual  combat  scenario  and  be  as  proficient  as  possible  in  the 
expected  events,  is  ultimately  restricted  to  his  currency  items  above  all  else.  This  pilot 
may  be  forced,  by  currency  tables,  to  spend  his  flight  time  leading  up  to  the  combat 
mission  completing  relatively  unnecessary  events  such  as  GPS  or  (Non-directional 
Beacon)  NDB  approaches,  knowing  full  well  that  there  are  no  GPS  or  NDB  approaches 
anywhere  in  the  combat  theater,  therefore  extremely  unlikely  to  be  needed. 

The  conclusion  to  be  gained  from  the  MAJCOM  comparisons  above  is  that  not  all 
continuation  training  currency  flight  events  that  are  currently  in  use  by  AMC  are 
absolutely  required  for  safe  execution  of  a  flying  mission.  Otherwise  ACC  aircraft  would 
be  experiencing  aviation  mishaps  at  a  higher  than  acceptable  rate.  The  next  discussion 
will  lead  to  weighing  the  more  important  and  critical  flight  events  in  a  quantitative 


manner  to  ensure  that  training  allocation  is  effectively  and  efficiently  accomplished. 
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Figure  2.3  Total  Flight  Events  by  MDS  (AFI 11-2MDS  Volume  1,  2006-2012) 
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Figure  2.4  Total  Flight  Events  by  MDS  (AFI 11-2MDS  Volume  1,  2006-2012) 


USAF  Currency  Requirements  vs  U.S.  Airline  Industry  (Part  121) 

U.S.  Air  Force  flight  currency  programs  differ  from  civil  airlines’  programs.  The 
Federal  Aviation  Administration  (FAA)  began  imposing  minimum  currency  requirements 
in  1974,  with  a  biannual  flight  review.  (Catton,  2009)  While  the  specific  requirements  of 
domestic  airlines  are  a  proprietary  agreement  between  each  airline  and  the  FAA,  the 
minimum  number  of  flight  requirements,  even  for  like  similar  aircraft  and  mission  sets,  is 
substantially  less  in  the  airline  sector  than  in  the  Air  Force.  As  a  regulatory  minimum, 
Part  121  of  the  Federal  Aviation  Regulation  mandates  that  no  less  than  3  takeoffs  and 
landings  in  the  previous  90  days  must  be  accomplished  for  a  pilot  to  serve  as  a  required 
flight  crewmember.  For  additional  flexibility,  the  FAA  has  granted  additional  leeway 
with  Part  121  airlines  by  accepting  the  substitution  of  proficiency  checks  for  required 
recurrent  flight  training.  (Federal  Aviation  Regulation,  Part  121,  2015) 


Weighted  Flight  Event  System 

In  order  to  begin  to  analyze  a  quantifiable  flight  event  system,  scores  must  be 
allocated  to  each  flight  event  based  on  applicable  criteria.  A  C-17  pilot  must  maintain 
currency  in  twenty-eight  flight  events  in  order  to  remain  on  “current”  status.  Otherwise, 
he  will  be  “non-current”  and  will  be  placed  on  a  restrictive  status  until  the  events  in 
which  he  went  non-current  for  are  completed  with  and  instructor  pilot  (IP).  (AFI  1 1-2C- 
17  Volume  1,  2012)  Each  event  is  treated  equally  important  to  maintain  currency  status 
and  squadron  leadership,  who  are  charged  with  ensuring  its  pilot  force  remains  current 
and  ready  to  complete  the  missions  assigned,  have  no  authority  or  flexibility  to  consider 
the  pilot  in  question’s  entire  situation  before  placing  the  pilot  on  restricted  status. 

For  example,  consider  a  senior  aircraft  commander  that  is  scheduled  to  fly  a  high- 
importance  operational  mission.  This  pilot  is  flies  an  average  of  approximately  1-1.5 
times  per  week  and  it  is  not  uncommon  to  become  non-current  in  a  flight  event  once  all 
nonnal  attrition  factors  such  as  maintenance,  weather,  and  TDYs  are  accounted  for.  In 
this  scenario,  squadron  leadership  has  virtually  no  other  option  than  to  place  the  pilot  on 
restricted  status  until  he  becomes  re-current  in  the  flight  event  that  he  went  overdue  for. 
The  subsequent  fall  out  of  this  scenario  is  potentially  more  impactful.  Because  the  senior 
aircraft  commander  has  gone  non-current,  squadron  leadership  has  no  choice  but  replace 
him  with  another  aircraft  commander  on  the  high-importance  mission.  The  replacement 
pilot  could  very  well  be  a  completely  new  and  unexperienced  but  fully  current  junior 
aircraft  commander.  In  effect,  the  squadron,  not  to  mention  the  mission  tasking 
authority,  has  lost  out  on  all  the  depth  and  experience  of  the  senior  aircraft  commander 
could  have  brought  to  the  mission  due  to  non-currency. 


What  is  missing  in  the  example  above  is  the  ability  of  the  squadron  to  consider 
the  senior  pilot’s  entire  situation  before  deeming  him  “non-current”  and  placing  him  on 
restricted  status.  For  starters,  the  squadron  leadership  should  know  which  event  the  pilot 
is  non-current  for  and  how  “important”  of  an  event  it  is  based  on  quantifiable  methods. 

Data  has  been  analyzed  and  constructed  to  apply  quantitative  scores  to  each  of  the 
twenty-eight  flight  events  for  an  experienced  C-17  aircraft  commander.  The  current  AFI 
1 1-2C-17  volume  one’s  minimum  required  flight  event  training  table  is  listed  below  for 


an  average  aircraft  commander: 


Aircraft  Commander  Currency 

Flight  Level  B  (Mid-Level) 

EVENT 

QTY 

Restriction 

1 

DAY  LOW  LEVEL 

1 

Fly  w/  IP 

2 

PRECAPPCH 

4 

Fly  w/  IP 

3 

NON-P  RECAP  PC  H 

3 

Fly  w/  IP 

4 

NDBAPPCH 

1 

Fly  w/  IP 

S 

THREAT  RESPONSE 

2 

Fly  w/  IP 

e 

HIGH  ALTTAC 

1 

Fly  w/  IP 

Per 

Semi 

7 

LOW  ALTTAC 

1 

Fly  w/  IP 

8 

CAT II  APPCH 

1 

Fly  w/  IP 

9 

RNAVAPPCH 

2 

Fly  w/  IP 

Annual 

10 

CIRCLE  APPCH 

2 

Fly  w/  IP 

11 

MSN  CPU  APPCH 

1 

Fly  w/  IP 

12 

LNDG  LZNIGHT 

1 

Fly  w/  IP 

13 

HVY  FFLDNG 

2 

Fly  w/  IP 

14 

HVY  FF  LONG  NT 

1 

Fly  w/  IP 

15 

NVG  LOW  LEVEL 

1 

Fly  w/  IP 

16 

NVG  INST APPCH 

1 

Fly  w/  IP 

17 

TAC  ARRIVAL 

2 

Fly  w/  IP 

18 

TAC  DEPT 

2 

Fly  w/  IP 

19 

NT  LANDING 

2 

Fly  w/  IP 

Per 

20 

LANDING  LZ 

2 

Fly  w/  IP 

Quarter 

21 

NVG  TAKEOFF 

2 

Fly  w/  IP 

22 

NVG  LDG 

1 

Fly  w/  IP 

23 

RECEIVER  ARAP  OFF 

2 

Fly  w/  IP 

24 

RECEIVER  AR  NIGHT 

2 

Fly  w/  IP 

Per 

Month 

25 

TAKEOFF 

1.6 

Fly  w/  IP 

26 

INST.  APPCH 

1.6 

Fly  w/  IP 

27 

LANDING 

1 

Fly  w/  IP 

Rolling 

28 

RECEIVER  AR 

1  per  60 
days 

Fly  w /  IP 

Table  2.1  Aircraft  Commander  Flight  Level  B  Currency  Requirements 
(AFI  11-2C-17  Volume  1,  2012) 


Each  of  the  above  twenty-eight  events  was  scored  on  three  criteria  categories: 
Complexity,  Probability,  and  Impact.  These  category  scores  are  used  as  the  primary 
quantitative  analysis  tool  to  give  a  cumulative  score  for  each  flight  event. 

Probability-Impact  Score 

In  the  civilian  market,  a  tool  has  been  developed  and  polished  over  time  to  aid 
decision  makers  in  the  risk  and  insurance  industry.  The  Probability-Impact  (PI)  Scale, 
Score,  or  Table  is  used  to  assign  a  quantifiable  category,  score,  or  stratification  to  a 
certain  event,  action,  or  inaction  in  order  to  provide  decision  makers  with  higher  fidelity 
information.  Probability  is  assessed  as  to  the  likelihood  that  an  event  will  occur  that  will 
create  a  hazard.  Impact  is  simply  the  magnitude  of  the  resulting  damage.  The  PI  table 
allows  a  company  or  entity  to  decide  which  risk  to  accept,  which  risk  to  avoid,  or  which 
risk  to  insure. 

The  goal  of  risk  management  and  the  PI  tool  is  not  to  avoid  or  eradicate  all  risk. 
This  would  be  a  practical  impossibility  in  the  real  world.  The  goal  of  the  PI  tool,  and  of 
risk  management,  is  to  seek  the  most  effective  and  cost  efficient  course  to  approach  risk. 
In  his  book  Risk  Analysis:  A  Quantitative  Guide,  author  David  Vose  explains  that  “the  P- 
I  table  offers  a  quick  way  to  visualize  the  importance  of  all  identified  risks  that  pertain  to 
a  project  [or  organization]”.  (Vose,  2008)  The  PI  scoring  system  also  provides  the 
quantitative  data  to  move  the  qualitative  assessments  of  risk  into  highly  adaptable  and 
analyzable  chart  and  graph  fonnats.  Vose  goes  on  to  highlight,  “it  is  clearly  important, 
however,  that  attention  is  focused  on  those  risks  that  pose  the  greatest  threat”.  This 
clarification  underlines  the  notion  that  not  all  risk,  and  in  the  case  of  this  research,  events 


with  risk,  should  be  scored,  treated,  and  restricted  with  the  same  weighting.  The 
expertise  in  Vose’s  text  would  beg  the  C-17  training  model  to  ensure  it  is  critically 
judging  each  event  and  adjust  the  training  planning  and  programming  as  necessary  to 
provide  the  most  efficient  risk  management  model  possible. 

The  PI  tool  only  analyzes  two  aspects  of  risk:  probability  and  impact.  For  the 
purposes  of  this  research,  the  PI  tool  will  be  adjusted  to  include  3  aspects  as  they  pertain 
to  C-17  flight  events.  This  research  will  focus  on  the  complexity  of  an  event,  the 
probability  of  its  occurrence,  and  the  risk  to  the  aircraft,  people,  and  mission  should  the 
event  not  be  correctly  performed. 

Skill  Score  (Ss) 

Flight  maneuvers  have  an  associated  level  of  complexity  based  on  how  difficult 
the  maneuver  is  to  complete.  For  the  purposes  of  this  research,  the  skill  score  is  defined 
both  by  the  level  of  skill  required  to  complete  the  maneuver  as  well  as  how  quickly  the 
skill  necessary  to  complete  the  maneuver  diminishes.  As  it  pertains  to  currency,  the 
diminishment  of  a  skill  is  the  primary  factor  when  detennining  how  long  between  events 
before  a  pilot  needs  to  re-accomplish  the  same  event  in  order  to  remain  minimally 
proficient.  The  Skill  Score,  when  analyzed  separately  from  the  entire  SPR  Score,  is  an 
average  score  between  two  separate  scores:  Skill  Required  and  Skill  Diminishment  Rate. 
In  general,  flight  events  that  are  more  complex  also  have  a  higher  skill  diminishment  rate. 
However,  in  order  to  capture  higher  fidelity  data,  two  separates  scores  are  obtained  in 
order  to  calculate  an  average  Skill  Score  that  will  be  utilized  in  the  total  SPR  Score.  A 
flight  event  that  takes  a  high  degree  of  quickly  diminishing  skills  is  weighted  with  a 
higher  Skill  Score  as  compared  to  an  event  that  takes  a  moderate  level  of  slowly 


diminishing  skills.  Upon  the  completion  of  a  given  flight  event,  a  pilot  is  assumed  to  be 
proficient  at  the  minimally  acceptable  level  to  complete  the  mission. 

Probability  Score  (Sp) 

Each  mission  or  training  sortie  is  unique  with  its  own  set  of  objective,  conditions, 
requirements,  and  personnel.  Therefore,  the  flight  events  and  their  order  for  most  sorties 
will  not  be  identical  to  any  other  sortie.  However,  when  the  data  is  compiled  over  a  large 
number  of  collections,  patterns  are  identifiable  and  certain  events  are  much  more  likely  to 
occur  than  others  on  an  average  flight.  Some  events  are  less  likely  due  to  the  resources 
and  complexity  of  the  coordination.  Air  refueling  is  a  prime  example.  Air  refueling 
sorties  are  significantly  more  expensive  due  to  the  added  aircraft  in  the  air  at  one  time. 
Therefore,  unless  a  requirement  exists  for  air  refueling,  it  is  not  seen  on  very  many  sorties 
as  compared  to  a  high  occurrence  event  such  as  takeoffs  and  landings.  Obviously,  high 
occurrence  events  such  as  these  will  occur  on  every  sortie  and  will  therefore  receive  a 
high  SP  Score. 

Ideally,  this  research  would  be  able  to  compile  and  quantify  the  historical 
averages  of  flight  events  over  a  statistically  significant  period  of  time.  However,  AMC 
does  not  currently  collect  data  to  analyze  the  exact  occurrence  rate  for  a  given  flight 
event.  In  the  absence  of  this  data,  for  this  research,  expert  opinions  will  be  sampled  as  to 
deduce  approximate  occurrence  rate  of  a  flight  event  relative  to  the  occurrence  rate  of 


each  of  the  other  events. 


Risk  Score  (Sr) 

The  nature  of  aviation  means  risk  exists  in  any  phase  of  flight.  However,  the  Risk 
Score  allows  stratification  amongst  flight  events  that  all  have  inherent  risk.  Risk  models 
in  the  civilian  sector  assess  risk  with  the  intent  to  address  the  risk  with  one  of  three 
options:  avoid,  assume,  or  mitigate  (insure)  risk.  The  assumption  for  this  research  is  that 
all  flight  events  that  are  deemed  necessary  to  maintain  aircraft  proficiency  are  at  a  risk 
level  that  is  acceptable  to  absorb  or  mitigate.  Again,  air  refueling  is  an  example  of  a  high 
risk  event  due  to  the  fact  that  two  aircraft  are  within  feet  of  each  other  while  traveling  at  a 
velocity  of  up  to  400  nautical  miles  per  hour.  A  small  miscalculation  or  mistaken  input  of 
control  can  easily  destroy  two  or  more  aircraft  along  with  several  crewmembers.  An 
example  of  an  event  that  results  in  a  lower  Risk  Score  would  be  one  that  either  has 
multiple  fail  safes  in  place  (either  policy  or  technology  placed),  moves  at  a  slow  enough 
pace  to  allow  for  more  time  for  the  pilot  to  mentally  process  the  situation. 

Skill-Probability-Risk  (SPR)  Score 

Once  the  data  from  the  Ss,  SP,  and  SR  has  been  calculated,  an  SPR  Score  can  be 
calculated.  The  SPR  Score  is  a  product  of  the  individual  four  subcategories.  This  score 
will  allow  the  necessary  and  appropriate  weighting  of  the  events  to  exist.  This  weighting 
is  the  primary  factor  in  the  final  stratification  of  the  events  and  their  associated  priority. 
“Landing  LZ  Night”,  for  example,  ranks  high  on  the  skill,  high  on  risk,  but  low  on 
probability.  Therefore,  “Landing  LZ  Night”  events  received  a  higher  SPR  Score  than 
“Takeoff’,  which  scored  very  low  on  skill,  but  very  high  on  probability,  and  moderately 
low  on  risk.  The  SPR  score  is  calculated  for  each  of  the  twenty-eight  flight  events  which 


will  result  in  a  1  through  n  rank  structure.  This  simple  calculation  is  a  major  step 
forward  for  squadron  leadership.  This  calculation  now  give  squadrons  quantifiable  data 
on  which  to  make  further  and  higher  fidelity  decisions  as  it  pertains  to  their  pilot  force’s 
actual  currency  status. 

Current  Methods  in  Quantitative  Aviation  Risk  Analysis 

The  civilian  aviation  industry,  specifically  the  FAA,  has  made  safety  and 
risk  management  an  official  program  of  record  since  the  1960s.  The  FAA  is  the  leader  in 
aviation  safety,  risk,  and  risk  management  fields.  The  US  Air  Force’s  safety  programs 
are  largely  derived  from  these.  Aviation  quantitative  risk  analysis  (QRA)  is  similar  to  the 
P-I  index  mentioned  above,  and  attempts  to  collect  subjective  assessments  of  exposure  to 
risk  probability  and  risk  severity,  then  to  subsequently  quantify  the  total  risk  exposure  to 
the  aviation  event.  The  FAA  has  developed  an  aviation  risk  assessment  matrix  that 
provides  flight  planners  and  pilots  with  the  current  method  of  quantifying  flight  risk  in  to 
categorical  results  of  acceptable,  acceptable  with  mitigation,  and  unacceptable. 
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Figure  2.5  FAA  Safety  Risk  Management  Matrix  (FAA,  2012) 


Thus  far,  aviation  risk  assessments  have  been  focused  on  general  flight  versus 
training  events.  This  approach  of  QRA  is  the  basis  for  the  methodology  in  this  paper  of 
the  SPR  Score.  SPR  is  a  continuation  of  the  QRA  matrix,  but  adds  and  uses  the 
numerical  score  of  each  event  to  prioritize  all  C-17  flight  events  verses  the  current 
method  of  risk  categorization.  Human  perception  is  the  foundation  to  the  FAA’s  QRA 
program.  (FAA,  Flight  Instructor  Training  Module,  Version  1.0)  Likewise,  the  SPR 
Score  is  a  factor  of  human  perception  toward  a  training  flight  event’s  subcategories. 
Furthennore,  the  FAA  cites  pilot  recency  of  experience  as  a  mitigating  factor  when 
addressing  aviation  risk.  (FAA,  Flight  Instructor  Training  Module,  Version  1.0) 
Therefore,  a  natural  research  progression,  and  a  purpose  of  this  paper  is  to  help  work 
toward  the  goal  of  understanding  how  much  recency  is  required  by  a  C-17  pilot  in  order 
to  mitigate  the  flight  event’s  inherent  risk,  based  on  the  SPR  subcategories  and  the  flight 
event’s  overall  relative  priority  score. 


III.  Methodology 


Chapter  Overview 

This  paper  utilized  several  mathematical  computations  in  order  to  calculate  a 
resulting  SPR  Score.  Interviews  with  several  C-17  mission  and  flight  training  experts 
have  provided  the  sufficient  data  to  the  research  in  order  to  calculate  relative  and 
quantifiable  scores  based  on  a  given  category,  to  include  skill,  probability  and  risk. 

Sample  Pool 

Assigning  a  quantifiable  score  to  each  flight  event  relies  heavily  upon 
subjective  perceptions  from  each  interviewee.  In  order  to  obtain  credible,  quantifiable 
data,  a  sample  size  that  was  large  enough  to  reasonably  derive  statistical  averages  was 
chosen.  This  research  interviewed  eight  C-17  evaluator  pilots.  The  average  flight  time  of 
the  pool  is  just  over  2700  hours.  All  interviewees  were  C-17  only  pilots,  with  seven 
being  evaluator  pilots  and  one  senior  instructor  pilot.  That  is,  none  of  the  interviewees 
have  flown  additional  MDS  aircraft  in  their  operational  careers  outside  of  pilot  training. 
Overall,  the  sample  size  of  the  interviewees  was  relatively  small  when  compared  to  larger 
research,  however,  the  sample  pool  was  specifically  chosen  due  to  their  expertise  in  the 
aircraft.  The  depth  of  experience  and  knowledge  in  the  sample  pool  negates  the  need  for 
a  large  sample.  Finally,  because  human  research  was  utilized,  an  exemption  to  human 
experimentation  was  sought  and  approved  in  accordance  with  32  CFR  219,  DoDD  3216.2 


and  AFI  40-402. 


Interview  Score  Sheet 


The  only  way  to  quantify  each  flight  event  is  to  assign  some  form  of 
numerical  score  to  each  item.  In  this  research,  each  interviewee  was  given  a  sheet  with 
each  of  the  28  C-17  flight  events.  The  events  were  listed  vertically  in  the  specific  order 
that  they  are  listed  in  the  Vol  1  as  to  not  mistakenly  communicate  any  inherent  or 
perceived  pre -ranking  or  grouping  among  the  events  on  the  score  sheet.  Each 
interviewee  was  given  instructions  to  individually  rank  each  item  1  through  28,  with  the 
lower  scores  indicative  of  the  most  skill  required  (S),  the  highest  probability  (P),  and  the 
most  risk  involved  (R).  Once  the  sheet  was  ranked  by  the  interviewee,  an  associated 
score  was  calculated  to  each  flight  event.  For  example,  if  an  event  was  ranked  the  #1 
priority  in  that  field,  it  was  assigned  the  maximum  score  of  28  points.  Conversely,  the 
#28  ranked  event  received  a  calculated  score  of  1  point  for  that  category.  This  ranking 
was  completed  for  4  categories:  Skill  Required,  Skill  Diminishment  Rate,  Probability  of 


Occurrence,  and  Risk. 


SKILL  REQUIRED  TO  COMPLETE 

SKILL  DIMINISHMENT  RATE 

YOUR 

RANKING 

SCORE 

EVENT 

YOUR 

RANKING 

SCORE 

EVENT 

DAY  LOW  LEVEL 

DAY  LOW  LEVEL 

PRECISION  APPCH 

PRECISION  APPCH 

NON-PREC  APPCH 

NON-PREC  APPCH 

NDB  APPCH 

NDB  APPCH 

THREAT  RESPONSE 

THREAT  RESPONSE 

HIGH  ALT  TAC 

HIGH  ALT  TAC 

LOW  ALT  TAC 

LOW  ALT  TAC 

CAT  II  APPCH 

CAT  II  APPCH 

RNAV  APPCH 

RNAV  APPCH 

CIRCLE  APPCH 

CIRCLE  APPCH 

MSN  CPU  APPCH 

MSN  CPU  APPCH 

LNDG  LZ  NIGHT 

LNDG  LZ  NIGHT 

HVY  FF  LDNG 

HVY  FF  LDNG 

HVY  FF  LDNG  NT 

HVY  FF  LDNG  NT 

NVG  LOW  LEVEL 

NVG  LOW  LEVEL 

NVG  INST  APPCH 

NVG  INST  APPCH 

NVG  LZ 

NVG  LZ 

TAC  DEPT 

TAC  DEPT 

NT  LANDING 

NT  LANDING 

LANDING  LZ 

LANDING  LZ 

NVG  TAKEOFF 

NVG  TAKEOFF 

NVG  LDG 

NVG  LDG 

RECEIVER  AR  AP  OFF 

RECEIVER  AR  AP  OFF 

RECEIVER  AR  NIGHT 

RECEIVER  AR  NIGHT 

TAKEOFF 

TAKEOFF 

INST.  APPCH 

INST.  APPCH 

LANDING 

LANDING 

RECEIVER  AR 

RECEIVER  AR 

Table  3.1  Skill  Score  Sheet 


PROBABILTY  OF  OCCURRENCE 

RISK  IF  NOT  PERFORMED  CORRECTLY 

YOUR 

RANKING 

SCORE 

EVENT 

YOUR 

RANKING 

SCORE 

EVENT 

DAY  LOW  LEVEL 

DAY  LOW  LEVEL 

PRECISION  APPCH 

PRECISION  APPCH 

NON-PREC  APPCH 

NON-PREC  APPCH 

NDB  APPCH 

NDB  APPCH 

THREAT  RESPONSE 

THREAT  RESPONSE 

HIGH  ALT  TAC 

HIGH  ALT  TAC 

LOW  ALT  TAC 

LOW  ALT  TAC 

CAT  II  APPCH 

CAT  II  APPCH 

RNAV  APPCH 

RNAV  APPCH 

CIRCLE  APPCH 

CIRCLE  APPCH 

MSN  CPU  APPCH 

MSN  CPU  APPCH 

LNDG  LZ  NIGHT 

LNDG  LZ  NIGHT 

HVY  FF  LDNG 

HVY  FF  LDNG 

HVY  FF  LDNG  NT 

HVY  FF  LDNG  NT 

NVG  LOW  LEVEL 

NVG  LOW  LEVEL 

NVG  INST  APPCH 

NVG  INST  APPCH 

NVG  LZ 

NVG  LZ 

TAC  DEPT 

TAC  DEPT 

NT  LANDING 

NT  LANDING 

LANDING  LZ 

LANDING  LZ 

NVG  TAKEOFF 

NVG  TAKEOFF 

NVG  LDG 

NVG  LDG 

RECEIVER  AR  AP  OFF 

RECEIVER  AR  AP  OFF 

RECEIVER  AR  NIGHT 

RECEIVER  AR  NIGHT 

TAKEOFF 

TAKEOFF 

INST.  APPCH 

INST.  APPCH 

LANDING 

LANDING 

RECEIVER  AR 

RECEIVER  AR 

Table  3.2  Probability  and  Risk  Score  Sheets 


SPR  Calculation 


The  SPR  Score  can  be  calculated  via  two  similar,  but  distinguishable  methods. 
Method  1  uses  a  sum  total  of  all  sub-scores  to  rank  the  events  by  total  overall  score. 
Method  2  uses  a  product  of  all  sub-scores  to  rank  the  events  by  total  overall  score.  This 
research  will  discuss  and  compare  any  ranking  discrepancies  between  the  two  methods, 
but  for  the  purposes  of  conclusion  and  recommendation,  Method  2  is  the  preferred 
analysis  method. 

The  SPR  Score  (Method  2  unless  otherwise  stated)  can  be  calculated  via  the 
following  method: 

SPR  =  (Ss  x  SP  x  SR)  x  100 

Equation  3.1  SPR  Score  Calculation 

Each  sub-category  (SS,  SP,  and  SR,  are  averages  of  the  subcategory’s  score  based 
on  the  total  score  assigned  by  the  interviewees.  For  example,  if  the  eight  interviewees 
scored  an  event  9,  8, 8, 4, 7, 8, 9, 7  respectively,  then  the  total  average  score  for  that  event 
would  be  7.5.  That  sub-score  would  then  be  multiplied  by  the  sub-scores  from  each  of 
the  other  four  categories.  That  product  would  then  be  multiplied  by  100  for  ease  of  use. 
Once  the  product  is  calculated  for  each  flight  event,  all  28  events  are  rank  ordered  from 
highest  score  to  lowest  score  for  analysis. 

EXAMPLE:  The  “DAY  LOW  LEVEL”  event  received  the  following  average 

scores  out  of  a  maximum  of  224  (28  points  x  8  interviewees): 

Skill  Required:  126 

Skill  Diminishment  Rate:  136 


Probability:  49 
Risk:  166 


Next,  the  average  score  was  divided  by  224  to  get  the  percentile  of  the  score  to 

detennine  how  strong  the  score  is.  The  closer  to  a  1.0  percentile  score,  the 

stronger  the  score  is.  The  following  percentiles  are  for  “DAY  LOW  LEVEL” 

Skill  Required:  .5625 
Skill  Dimini shment  Rate:  .61 
Probability:  .22 
Risk:  .74 

To  obtain  a  final  SPR  score,  each  category  is  multiplied  be  each  other,  and  then 
multiplied  by  lOOfor  ease  of  use.  The  maximum  theoretical  possible  score  is  100. 

(.5625  x  .61  x  .22  x  .74)  x  100  =  5.54  SPR  Score 


To  obtain  a  ranking,  aka  “RANK”,  the  SPR  Scores  are  arranged  from  highest  to  lowest  to 
calculate  the  associated  ranking.  The  highest  SPR  Score  receives  a  rank  of  1,  and  the 
lowest  SPR  Score  receives  a  rank  of  28.  For  some  of  the  comparison  in  the  analysis 
section,  a  ranking  was  converted  to  a  “RANK-SCORE”.  This  is  calculated  by 
subtracting  the  “RANK”  from  a  value  of  29.  Therefore,  a  “RANK”  of  1  would  have  an 
associated  “RANK-SCORE”  of  28;  the  maximum  possible  value. 


IV.  Analysis  and  Results 


Chapter  Overview 

The  data  collection  from  the  interviewees  allows  qualitative  subjectivity  to  be 
quantified  and  stratified.  The  data  allows  the  research  to  compare  and  contrast  the 
expert-supplied  data  with  that  from  the  Vol  1  ’s  training  tables.  The  data  was  stratified 
based  on  relative  importance;  those  events  that  scored  highest  on  the  SPR  score  are 
deemed  to  be  more  important  based  on  the  expert  opinion. 

Skill  Required 

Each  event  was  ranked  based  on  the  skill  required  for  the  event  by  the 
expert  panel.  A  flight  event  that  requires  a  large  amount  of  skill  will  be  ranked  relatively 
high  (low  numerical  ranking),  and  will  have  a  relatively  high  associated  score.  The 
following  table  shows  the  flight  event  ranking  and  score  totals,  in  descending  order.  The 
maximum  possible  rank  or  score  is  224  and  the  theoretical  maximum  percentile  is  1.00: 


SKILL  REQUIRED 

RANK 

SCORE 

PERCENTILE 

EVENT 

16 

216 

0.9643 

RECEIVER  ARAP  OFF 

16 

216 

0.9643 

RECEIVER  AR  NIGHT 

31 

201 

0.8973 

NVG  LZ 

36 

196 

0.8750 

RECEIVER  AR 

43 

189 

0.8438 

LNDGLZ  NIGHT 

49 

183 

0.8170 

LANDING  LZ 

73 

159 

0.7098 

HVY  FF  LDNG  NT 

88 

144 

0.6429 

HIGH  ALTTAC 

93 

139 

0.6205 

HVY  FF  LDNG 

93 

139 

0.6205 

NVGLDG 

98 

134 

0.5982 

LOW  ALTTAC 

104 

128 

0.5714 

NVG  LOW  LEVEL 

106 

126 

0.5625 

DAY  LOW  LEVEL 

124 

108 

0.4821 

CIRCLE  A PPCH 

127 

105 

0.4688 

NT  LANDING 

138 

94 

0.4196 

NVG  TAKEOFF 

140 

92 

0.4107 

NDBAPPCH 

149 

83 

0.3705 

TAC  DEPT 

151 

81 

0.3616 

NVG  INST  APPCH 

152 

80 

0.3571 

THREAT  RESPONSE 

155 

77 

0.3438 

CAT  II  APPCH 

163 

69 

0.3080 

LANDING 

168 

64 

0.2857 

MSN  CPU  APPCH 

171 

61 

0.2723 

NON-PREC  APPCH 

173 

59 

0.2634 

RNAV  APPCH 

194 

38 

0.1696 

INST.  APPCH 

198 

34 

0.1518 

PREC  ISION  APPCH 

199 

33 

0.1473 

TAKEOFF 

Table  4.1  Skill  Required  Score  Results 


Skill  Diminishment  Rate 

Each  event  was  ranked  based  on  the  rate  in  which  the  skills  required 
diminish  over  time.  A  flight  event  that  has  a  relatively  quick  skill  diminishment  rate  will 
be  ranked  relatively  high  (low  numerical  ranking),  and  will  have  a  relatively  high 
associated  score.  The  following  table  shows  the  flight  event  ranking  and  score  totals  in 
descending  order.  The  maximum  possible  rank  or  score  is  224  and  the  theoretical 
maximum  percentile  is  1.00: 


SI 


RANK 

SCORE 

PERCENTILE 

EVENT 

12 

220 

0.98 

RECEIVER  ARAP  OFF 

16 

216 

0.96 

RECEIVER  AR  NIGHT 

29 

203 

0.91 

RECEIVER  AR 

31 

201 

0.90 

NVGL2 

44 

188 

0.84 

LNDGLZ  NIGHT 

50 

182 

0.81 

LANDING  LZ 

82 

150 

0.67 

HVY  FF  LDNG  NT 

91 

141 

0.63 

NVG  LDG 

96 

136 

0.61 

DAY  LOW  LEVEL 

97 

135 

0.60 

HIGH  ALTTAC 

100 

132 

0.59 

HVY  FF  LDNG 

101 

131 

0.58 

LOW  ALTTAC 

102 

130 

0.58 

NVG  LOW  LEVEL 

118 

114 

0.51 

NT  LANDING 

126 

106 

0.47 

CIRCLE  APPCH 

136 

96 

0.43 

NVG  TAKEOFF 

140 

92 

0.41 

THREAT  RESPONSE 

141 

91 

0.41 

TAC  DEPT 

148 

84 

0.38 

NDBAPPCH 

154 

78 

0.35 

MSN  CPU  APPCH 

154 

78 

0.35 

NVG  INST APPCH 

154 

78 

0.35 

LANDING 

168 

64 

0.29 

NON-PREC  APPCH 

170 

62 

0.28 

RNAV  APPCH 

180 

52 

0.23 

CAT  II  APPCH 

194 

38 

0.17 

TAKEOFF 

197 

35 

0.16 

INST.  APPCH 

206 

26 

0.12 

PRECISION  APPCH 

Table  4.2  Skill  Diminishment  Rate  Score  Results 


Probability  of  Occurrence 

Each  event  was  ranked  based  on  the  probability  that  the  event  will  be 
required  for  mission  accomplishment  on  any  given  flight.  Expert  interviewees  did  not 
account  for  those  events  that  are  completed  solely  for  the  purpose  of  fulfilling  Vol  1 
requirements.  A  flight  event  that  has  a  relatively  high  probability  will  be  ranked 
relatively  high  (low  numerical  ranking),  and  will  have  a  relatively  high  associated  score. 
The  following  table  shows  the  flight  event  ranking  and  score  totals  in  descending  order. 


The  maximum  possible  rank  or  score  is  224  and  the  theoretical  maximum  percentile  is 


1.00: 


RANK 

SCORE 

PERCENTILE 

EVENT 

11 

221 

0.99 

TAKEOFF 

15 

217 

0.97 

LANDING 

31 

201 

0.90 

PRECISION  A PPCH 

33 

199 

0.89 

INST.  APPCH 

50 

182 

0.81 

NON-PREC  APPCH 

61 

171 

0.76 

NT  LANDING 

94 

138 

0.62 

LOW  ALT TAC 

98 

134 

0.60 

RNAV  APPCH 

100 

132 

0.59 

CAT  II  APPCH 

101 

131 

0.58 

NVGLDG 

103 

129 

0.58 

HVY FF  LDNG 

104 

128 

0.57 

NVG  TAKEOFF 

106 

126 

0.56 

HIGH  ALT  TAC 

106 

126 

0.56 

TAC  DEPT 

119 

113 

0.50 

CIRCLE  APPC  H 

120 

112 

0.50 

HVY  FF  LDNG  NT 

144 

88 

0.39 

THREAT  RESPONSE 

145 

87 

0.39 

MSN  CPU  APPCH 

146 

86 

0.38 

RECEIVER  AR 

152 

80 

0.36 

NVG  INST  APPC  H 

159 

73 

0.33 

LANDING  LZ 

162 

70 

0.31 

RECEIVER  AR  NIGHT 

163 

69 

0.31 

LNDGLZ  NIGHT 

180 

52 

0.23 

NVG  LZ 

180 

52 

0.23 

RECEIVER  AR  AP  OFF 

183 

49 

0.22 

DAY  LOW  LEVEL 

186 

46 

0.21 

NDB  APPCH 

196 

36 

0.16 

NVG  LOW  LEVEL 

Table  4.3  Probability  Score  Results 


Risk 

Each  event  was  ranked  based  on  the  inherent  risk  associated  with  the  event 
if  the  maneuver  is  not  performed  correctly.  This  category  acknowledges  that  in  any  flight 
event,  the  risk  of  death  and  aircraft  loss  is  possible.  Therefore,  events  that  have  fewer 
redundancies  or  time  to  react,  for  example,  are  considered  more  risky  than  others.  A 
flight  event  that  has  relatively  risk  will  be  ranked  relatively  high  (low  numerical  ranking), 
and  will  have  a  relatively  high  associated  score.  The  following  table  shows  the  flight 


event  ranking  and  score  totals  in  descending  order.  The  maximum  possible  rank  or  score 
is  224  and  the  theoretical  maximum  percentile  is  1.00: 


RANK 

SCORE 

PERCENTILE 

EVENT 

26 

206 

0.92 

RECEIVER  ARAP  OFF 

26 

206 

0.92 

RECEIVER  AR  NIGHT 

38 

194 

0.87 

NVG  LZ 

40 

192 

0.86 

RECEIVER  AR 

44 

1 88 

0.84 

LNDGLZ  NIGHT 

58 

174 

0.78 

LANDING  L2 

62 

170 

0.76 

HVY  FF  LDNG  NT 

66 

166 

0.74 

DAY  LOW  LEVEL 

83 

149 

0.67 

NVG  LDG 

86 

146 

0.65 

NVG  LOW  LEVEL 

90 

142 

0.63 

HVY  FF  LDNG 

104 

128 

0.57 

THREAT  RESPONSE 

106 

126 

0.56 

NT  LANDING 

121 

111 

0.50 

NVG  TAKEOFF 

130 

102 

0.46 

LANDING 

133 

99 

0.44 

LOW  ALT TAC 

143 

89 

0.40 

TAKEOFF 

150 

82 

0.37 

NVG  INST  APPCH 

150 

82 

0.37 

TAC  DEPT 

154 

78 

0.35 

CAT  II  APPCH 

160 

72 

0.32 

HIGH  ALT  TAC 

164 

68 

0.30 

CIRCLE  APPCH 

177 

55 

0.25 

NDB  APPCH 

181 

51 

0.23 

RNAV  APPCH 

183 

49 

0.22 

MSN  CPU  APPCH 

187 

45 

0.20 

PRECISION  APPCH 

192 

40 

0.18 

NON-PREC  APPCH 

194 

38 

0.17 

INST.  APPCH 

Table  4.4  Risk  Score  Results 


Cross-Comparison  of  SPR  Sub-Scores 

When  each  sub-category  is  analyzed  individually,  it  produces  a  relatively  linear 
pattern  of  scores  ranging  from  the  high  .9’s  range  to  the  low  .  1  ’s  range.  Each  category  does  not 
communicate  very  much  information  until  it  is  compared  with  the  other  sub-categories  amidst 
the  backdrop  of  the  SPR  Percentile  Ranking.  Figure  4.5  illustrates  all  4  sub-categories  when 
ranked  in  the  SPR  ranking  based  on  percentile. 


Interestingly,  Skill  required,  Skill  Diminish  Rate,  and  Risk  sub-categories 
demonstrate  a  linear  relationship  when  compared  to  SPR  score,  as  well  as  they  are  clearly 
associated  with  each  other.  Probability,  on  the  other  hand,  demonstrates  no  relationship  with 
SPR  score. 


Figure  4.1Cross  Comparison  of  Skill,  Probability,  and  Risk  Scores 

SPR  Score 

Once  the  subcategories  were  tallied,  an  SPR  score  was  calculated  in  order  to  rank  all  28 
events  by  percentile  score.  The  ranking  is  listed  in  figure  4.  6  below.  Figure  4.  X  illustrates  the 
events’  SPR  score  values  on  a  plotting  chart.  This  chart  is  important  because  it  highlights  any 
relationships  among  the  scores  relative  to  each  other.  From  this  chart  it  can  be  viewed  that  all  28 
events  scored  in  1  of  3  distinct  groupings  .  For  the  purpose  of  this  research,  the  3  groups,  or 
categories,  will  be  “High”,  “Medium”,  and  “Low”.  High  events  are  those  that  scored  above  20 
points.  Medium  events  are  between  6  and  20  points.  Low  events  scored  below  6  points. 


SPR  SCORE  VALUE 


30.00 


1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28 


Figure  4.2  SPR  Score  Value 


RANK 

SPR  SCORE 

EVENT 

1 

26.72 

RECEIVER  AR  NIGHT 

2 

26.10 

RECEIVER  AR 

3 

20.22 

RECEIVER  AR  AP  OFF 

4 

18.31 

LNDG  LZ  NIGHT 

5 

18.04 

HVY FF  LDNG  NT 

6 

16.80 

LANDING  LZ 

7 

16.19 

NVG  LZ 

8 

15.19 

NVG LDG 

9 

13.35 

HVY  FF  LDNG 

10 

10.24 

NT  LANDING 

11 

933 

LOW  ALT  TAC 

12 

7.00 

HIGH  ALT  TAC 

13 

5.54 

DAY  LOW  LEVEL 

14 

5.09 

NVG  TAKEOFF 

15 

4.73 

LANDING 

16 

3.49 

CIRCLE  APPCH 

17 

3.47 

NVG  LOW  LEVEL 

18 

3.29 

THREAT  RESPONSE 

19 

3.10 

TAC  DEPT 

20 

1.65 

NVG  INST  APPCH 

21 

1.64 

CAT  II  APPCH 

22 

1.13 

NON-PREC  APPCH 

23 

0.99 

RNAV  APPCH 

24 

0.98 

TAKEOFF 

25 

0.85 

MSN  CPU  APPCH 

26 

0.78 

NDB  APPCH 

27 

0.40 

INST.  APPCH 

28 

0.32 

PRECISION  APPCH 

Table  4.5  Events  Ranked  by  SPR  Score 


SPR  Rank-Score  vs  Vol  1  Rank-Score 


By  analyzing  the  data  points,  it  is  clear  to  see  that  there  are  significant  discrepancies  between  the 
SPR  Rank-Score  and  the  Vol  1  Rank-Score.  Figure  4.8  illustrates  the  differences  in  scores  between  the 
two  processes.  Ideally,  the  two  score  sets  would  be  much  more  highly  aligned  with  each  other.  Events 
that  the  SPR  data  deemed  the  most  important  (upper  left  of  the  chart)  were  scored  equally  unimportant 
by  Vol  1  standards.  Similarly,  events  that  were  scored  very  low  by  SPR  data  were  conversely  scored 
very  high  by  Vol  1  tables.  The  extreme  lower  left  and  extreme  upper  right  of  the  chart  are  areas  of 
major  discrepancy  between  the  two  processes  and  potentially  provide  the  areas  most  ripe  for 
reconciliation. 


Figure  4.3  SPR  Rank-Score  vs  Vol  1  Rank-Score 


Figure  4.9  below  summarizes  the  scoring  associated  with  each  flight  event.  Take  note  that  the 
SPR  score  and  the  Vol  1  scores  are  not  on  the  same  scale,  therefore  cannot  be  directly  compared.  This  is 


the  basis  for  the  Rank-Score  scale  used  above. 


Maximum  Number  of  Days  in  Applicable  Currency  Period 

Minimum  Number  of  Events  Required  in  Period 

*  Lower  Vol  1  Calculation  score  indicates  a  higher  Vol  1  priority 

Equation  4.1  Vol  1  Score  Calculation 


Event 

SPR  Score 

VOL  1  Score 

Skill  Req 

Skill  Dim 

Prob 

Risk 

RECEIVER  AR  NIGHT 

26.40 

2.27273 

0.96 

0.96 

0.31 

0.92 

RECEIVER  AR 

26.02 

1.72414 

0.88 

0.91 

0.38 

0.86 

RECEIVER  AR  AP  OFF 

20.20 

2.27273 

0.96 

0.99 

0.23 

0.92 

LNDGLZ  NIGHT 

18.46 

0.56180 

0.84 

0.84 

0.31 

0.84 

HVY FF  LDNG  NT 

18.07 

0.56180 

0.71 

0.67 

0.50 

0.76 

LANDING  LZ 

17.59 

2.27273 

0.84 

0.81 

0.33 

0.78 

NVGLZ 

16.16 

1.13636 

0.90 

0.90 

0.23 

0.87 

NVGLDG 

15.19 

1.13636 

0.62 

0.63 

0.58 

0.67 

HVY  FF  LDNG 

13.38 

1.12360 

0.62 

0.59 

0.58 

0.63 

NT  LANDING 

10.18 

2.27273 

0.47 

0.51 

0.76 

0.56 

LOW  ALT  TAC 

9.46 

0.56180 

0.60 

0.58 

0.62 

0.44 

HIGH  ALT TAC 

6.91 

0.56180 

0.64 

0.60 

0.56 

0.32 

DAY  LOW  LEVEL 

5.59 

0.56180 

0.56 

0.61 

0.22 

0.74 

NVG  TAKEOFF 

5.14 

2.27273 

0.42 

0.43 

0.57 

0.50 

LANDING 

4.81 

3.57143 

0.31 

0.35 

0.97 

0.46 

CIRCLE  APPCH 

3.40 

1.12360 

0.48 

0.47 

0.50 

0.30 

NVG  LOW  LEVEL 

3.45 

0.56180 

0.57 

0.58 

0.16 

0.65 

THREAT  RESPONSE 

3.25 

1.12360 

0.36 

0.41 

0.39 

0.57 

TAC  DEPT 

3.15 

2.27273 

0.37 

0.41 

0.56 

0.37 

NVG  INST  APPCH 

1.69 

0.56180 

0.36 

0.35 

0.36 

0.37 

CAT  II  APPCH 

1.63 

0.56180 

0.34 

0.23 

0.59 

0.35 

NON-PREC  APPCH 

1.15 

1.68539 

0.27 

0.29 

0.81 

0.18 

RNAV  APPCH 

1.02 

1.12360 

0.26 

0.28 

0.60 

0.23 

TAKEOFF 

0.99 

5.71429 

0.15 

0.17 

0.99 

0.40 

MSN  CPU  APPCH 

0.86 

0.56180 

0.29 

0.35 

0.39 

0.22 

NDB  APPCH 

0.82 

0.56180 

0.41 

0.38 

0.21 

0.25 

INST.  APPCH 

0.41 

5.71429 

0.17 

0.16 

0.89 

0.17 

PRECISION  APPCH 

0.33 

2.24719 

0.15 

0.12 

0.90 

0.20 

Table  4.6  Event  Score  Summary 


Investigative  Questions  Answered 


What  is  the  current  AMC  C-l  7  training  model? 

It  has  been  established  that  the  current  AMC  C-l 7  training  model  is  solely 
based  on  currency  for  each  of  the  28  selected  events.  While  it  is  unclear  exactly 
how  the  number  of  event  repetitions  per  time  period  was  originally  composed, 
this  research  provides  leadership  with  high-fidelity  and  quantifiable  data  on  which 
to  create  training  tables  that  more  appropriately  align  with  the  skill,  probability, 
and  risk  associated  with  each  event. 

How  does  it  compare  to  the  rest  of  the  Air  Force  MDSs? 

The  C-l 7  training  model  is  in  line  with  the  whole  of  AMC’s  training 
model.  AMC  has  been  shown  to  require  substantially  more  flight  events  per  time 
period  than  the  rest  of  the  operational  Air  Force.  The  SPR  data  can  be  the  first 
step  in  better  prioritizing  training  events  so  that  valuable  time  and  resources  can 
be  better  allocated  to  the  appropriate  flight  events. 

How  do  US AF  currency  requirements  differ  from  U.S.  civilian  airlines? 

The  FAA  establishes  the  minimum  number  of  events  within  a  given  time 
period  for  U.S.  Airlines  in  the  Federal  Aviation  Regulation  (FAR)  Section  121. 
The  FAR’s  minimums  are  substantially  less  than  that  of  any  AMC  Volume  1. 

The  FAR’s  only  recency  requirement  is  for  a  pilot  to  accomplish  3  takeoffs  and  3 
landings  within  the  preceding  90  days.  This  equates  to  12  flight  events  per  180 


days,  which  is  the  AMC  standard  period  for  its  Volume  1  regulations.  All  AMC 
aircraft  have  between  31  and  36  flight  events,  not  including  “mission”  flight 
events  (tactical  maneuvers,  airdrop,  etc.)  in  the  same  period.  (AFI  1 1-2MDS 
Volume  1,  2006-2012)  This  equates  to  AMC  pilots  being  responsible  for  300% 
of  the  like  events  that  airline  pilots  are  required  to  perform.  If  “mission”  events 
are  included,  AMC  pilots  are  required  to  complete  as  much  as  600%  of  those 
required  of  airline  pilots.  (AFI  1 1-2MDS  Volume  1,  2006-2012) 

Are  all  C-l  7  flight  events  equal? 

Even  though  the  Vol  1  perceives  all  C-l 7  flight  events  as  being  equal  in 
importance  with  respect  to  currency  status,  the  SPR  results  argue  that  the 
qualitative  data  provided  by  the  expert  interviews  indicate  that  the  events  are  far 
from  equal  and  should  be  addressed  as  such  when  developing  training  plans  and 
assessing  currency  or  proficiency. 

Can  C-l  7  training  events  be  quantified  and  stratified? 

This  research  has  concluded  that  C-l 7  training  events  can  be  quantified 
and  stratified.  While  the  SPR  model  is  only  one  approach  to  the  process,  it 
demonstrates  that  qualitative  infonnation  from  the  experts  in  the  C-l 7  pilot  force 
can  be  collected  and  quantified  to  provide  useful  tools  to  the  commander  and 
AMC  leadership. 

What  flexibility  is  gained  at  the  squadron,  group,  and  MAJCOM  level? 

This  research  has  given  AMC  leadership  a  significant  opportunity  to 
empower  squadrons,  relieve  the  pilot  force  of  relatively  unnecessary  but 
restrictive  flight  events,  and  save  the  Air  Force  valuable  flight  and  man  hours. 


Not  to  be  outdone,  this  process  can  provide  the  Combatant  Commander  with  more 
proficient  and  higher  caliber  pilots  at  a  reduced  cost. 

Feasibility  (Hammer,  M.  and  Champy,  J.,  2001) 

a.  Technical  -  This  data  can  be  easily  used  or  the  process  replicated  on  a  large 
scale.  Additionally,  this  research  only  amplifies  the  abilities  and  decision 
making  of  AMC  leadership  by  providing  the  stratified  priority  based  on 
actual  expert  data.  The  overall  process  can  easily  be  implemented  by  AMC 

b.  Economic  -  There  is  relatively  no  costs  to  replicate  or  implement  this 
process  or  program. 

c.  Cultural  -  Due  to  the  argument  that  this  research  can  birth  a  program  that 
empowers  squadrons,  reduces  costs,  reduces  restrictions  on  pilots,  and 
provides  a  better  product  to  the  fight,  no  pushback  can  be  predicted  based 


on  the  results  of  this  research. 


V.  Conclusions  and  Recommendations 


Chapter  Overview 

The  research  and  subsequent  data  from  this  paper  can  provide  a  significant  impact 
to  the  C-17  training  model.  This  chapter  will  focus  on  the  effects,  improvements,  and 
savings  that  can  be  generated  should  this  data  and  its  recommendations  be  utilized. 

Fiscal  realities  within  the  Department  of  Defense  are  forcing  Ainnen  to  seek  out  new  and 
innovative  ways  to  reduce  costs  and  improve  productivity  within  a  smaller  and  smaller 
force. 

Conclusions  of  Research 

The  conclusion  of  this  research  is  that  qualitative  and  subjective  data  from  a 
significant  sample  set  can  provide  a  quantitative  means  to  analyze  and  apply  C-17 
training  events.  Under  the  current  model,  the  Vol  1  and  the  experts  in  this  research  are 
in  a  significant  disagreement  between  which  training  events  are  most  important  based  on 
required  skill,  probability  of  the  event  occurring  on  a  mission,  and  the  risk  if  the  event  is 
not  perfonned  correctly. 

Furthermore,  this  research  highlights  relatively  drastic  differences  between 
MAJCOMs  when  determining  the  minimum  currency  training  events  for  a  pilot.  AMC 
requires  significantly  more  currency  events  in  a  training  period  than  that  of  ACC  and 
AFSOC,  even  among  like  airframes. 


Significance  of  Research 

In  practice,  this  research  and  its  implementation  are  relatively  simple;  simple  in 
calculation,  administration,  as  well  as  execution.  For  the  first  time  ever,  AMC  now  has 
quantifiable  and  stratifiable  means  to  assess  and  administer  its  training  events. 

As  mentioned  before,  currently,  all  training  events  are  treated  as  equally 
important  with  respect  to  currency.  A  pilot  that  is  non-current  for  “Receiver  AR  Night”, 
which  was  ranked  the  most  important  via  SPR  Score,  is  just  as  ineligible  to  fly  a  mission 
as  the  pilot  that  is  non-current  for  precision  approach,  which  is  ranked  as  least  important 
based  on  SPR  Score.  Under  current  regulations,  this  pilot,  even  though  he  may  be  the 
most  experienced  and  capable  pilot  in  the  squadron  is  not  legally  permitted  to  complete 
this  mission.  The  Vol  1  does  allow  the  Operations  Group  Commander,  an  0-6,  waiver 
authority  for  the  currency  item.  This  policy  does  not  provide  any  guidance  for  AMC’s 
intent  on  when  or  how  the  waiver  process  is  intended  to  be  executed. 

Should  this  model  be  used,  it  is  possible  for  AMC  to  determine  a  threshold  of 
authority  in  which  squadrons  and  groups  have  clear  guidance  on  what  can  be  waived,  by 
which  level  of  authority,  and  how  often  it  should  be  waived.  For  example,  if  we  remain 
with  the  simplified  High,  Medium,  and  Low  classes  of  events  as  was  mentioned  earlier,  it 
is  feasible  for  AMC  to  delegate  waiver  authority  for  Low  category  events  to  Squadron 
Commander,  perhaps  Medium  category  waiver  authority  to  the  Operations  Group 
Commander,  and  High  category  events  are  only  waiverable  by  MAJCOM.  Squadron 
commanders  could  then  look  at  the  pilot  more  holistically  when  determining  if  the  pilot 
should  receive  the  waiver.  If  the  pilot  happens  to  be  a  highly  experienced  evaluator  pilot 
with  3500  hours  in  the  C-17,  it  makes  complete  sense  to  waive  an  NDB  approach  non- 


currency  status  so  that  the  pilot  can  fill  a  mission  for  a  Combatant  Commander,  or  fill  a 
high-priority  alert  to  move  the  President’s  equipment  and  personnel  for  a  high-impact 
political  summit. 

Additionally,  cost  savings  possibilities  are  abundant.  First,  the  pilot  that  receives 
a  waiver  immediately  realizes  cost  savings  by  not  having  to  fly  additional  training  hours 
to  regain  currency.  Additional  instructor  pilot  man-hours  are  not  wasted  by  supervising  a 
highly  capable  pilot  regain  the  currency  on  an  item  that  scored  very  low  on  the  SPR 
Score,  therefore  may  be  a  relatively  unimpactful  flight  event. 

In  the  future,  it  could  be  highly  beneficial  for  AMC  to  recognize  those  flight 
events  that  are  relatively  unimportant  for  currency  and  could  be  removed  from  the 
currency  table.  This  could  lead  AMC’s  training  tables  to  more  closely  mirror  those  in  the 
rest  of  the  operational  flying  Air  Force.  Reducing  training  events  will  clearly  reduce 
costs  associated  with  those  training  sorties  that  are  solely  generated  to  maintain  or  regain 
currency.  Those  flight  hours  could  be  harvested  for  savings,  or  possibly  administered  in 
a  training  plan  that  will  increase  proficiency  and  capability  in  flight  events  that  are 
deemed  necessary  by  the  squadron  training  office  or  leadership. 

In  summary,  the  model  that  this  research  makes  possible  reduces  costs,  increases 
capability,  reduces  overhead,  and  reduces  man  hours. 

Recommendations  for  Action 

It  is  recommended  that  the  training  model  that  this  research  allows  and 
recommends  be  reviewed  by  AMC.  Additionally,  a  pilot  program  should  be  established  at 
a  C-17  Wing  in  order  to  study  the  effects  of  such  a  program  on  the  squadron.  These 


results  will  assist  in  detennining  whether  the  program  be  adjusted  or  administered  AMC- 
wide. 


Recommendations  for  Future  Research 

This  research  allows  a  training  model  that  opens  up  future  possibilities.  There  are 
several  suggestions  for  future  research  based  on  the  data  collected  in  this  thesis: 

1 .  Conduct  similar  studies  in  other  AMC  airframes  to  detennine  if  the 
discrepancy  between  the  Vol  1  and  the  expert  judgement  is  limited  to  the  C- 
17. 

2.  Conduct  studies  of  ACC  flying  units  to  detennine  what  best  practices  and 
efficiencies  could  be  gained  by  AMC  by  adjusting  a  training  model  more 
aligned  with  CAF  units. 

3.  Utilize  the  quantitative  data  from  this  or  follow-on  studies  to  develop  an 
optimization  tool  that  could  be  used  by  squadrons  to  ensure  the  highest 
priority  pilots  are  flying  the  most  appropriate  training  lines,  while  conducting 
the  highest  priority  flight  events  that  will  most  support  the  Combatant 
Commander. 


Summary 

In  conclusion,  AMC  currently  maintains  a  training  model  for  the  C-17  that  does  a 
remarkable  job  at  ensuring  that  every  pilot  that  flies  a  mission  has  completed  at  least  a 
minimum  number  of  flight  events  within  a  given  period.  This  model  has  worked  very 


well  at  exposing  each  pilot  to  a  myriad  of  events  that  he  could  face  while  on  the  mission 
to  a  seemingly  limitless  number  of  airfields  and  environments. 

However,  this  research  demonstrates  a  possible  surplus  in  training  events  by 
AMC  units  when  compared  to  their  CAF  counterparts  who  are  also  deemed  safe  and 
equipped  to  fly.  Additionally,  this  research  provides  the  means  to  quantify  expert 
subjective  opinion  that  can  either  support  or  challenge  the  training  model  and  tables  that 
are  provided  in  regulatory  guidance,  specifically  the  1 1-2MDS  Volume  1. 

The  quantitative  data  collected  in  this  research  has  highlighted  significant 
discrepancies  between  expert  opinion  and  Vol  1  guidance  regarding  the  relative 
significance  of  each  of  28  selected  strategic  flight  events.  These  discrepancies  warrant  at 
least  further  investigation  if  not  full  implementation  resulting  in  a  major  overhaul  in  the 
C-17  training  program  which  could  easily  result  in  major  cost  savings  and  capability 


increases. 


Appendix  A 


SPR 

RANK 

SCORE 

EVENT 

1 

28 

RECEIVER  AR  NIGHT 

2 

27 

RECEIVER  AR 

3 

26 

RECEIVER  ARAP  OFF 

4 

25 

LNDG  LZ  NIGHT 

5 

24 

HVY  FF  LDNG  NT 

6 

23 

LANDING  L2 

7 

22 

NVG  LZ 

8 

21 

NVG LDG 

9 

20 

HVY  FF  LDNG 

10 

19 

NT  LANDING 

11 

18 

LOW  ALT  TAC 

12 

17 

HIGH  ALT  TAC 

13 

16 

DAY  LOW  LEVEL 

14 

15 

NVG  TAKEOFF 

15 

14 

LANDING 

16 

13 

CIRCLE  APPCH 

17 

12 

NVG  LOW  LEVEL 

18 

11 

THREAT  RESPONSE 

19 

10 

TAC  DEPT 

20 

9 

NVG  INST  APPCH 

21 

8 

CAT  II  APPCH 

22 

7 

NON-PREC  APPCH 

23 

6 

RNAV  APPCH 

24 

5 

TAKEOFF 

25 

4 

MSN  CPU  APPCH 

26 

3 

NDB  APPCH 

27 

2 

INST.  APPCH 

28 

1 

PRECISION  APPCH 

VOLUME  1 

RANK 

SCORE 

EVENT 

1 

28 

TAKEOFF 

2 

27 

INST.  APPCH 

3 

26 

LANDING 

4 

25 

TAC  DEPT 

5 

24 

NT  LANDING 

6 

23 

LANDING  LZ 

7 

22 

NVG  TAKEOFF 

8 

21 

RECEIVER  AR  AP  OFF 

9 

20 

RECEIVER  AR  NIGHT 

10 

19 

PREC  APPCH 

11 

18 

RECEIVER  AR 

12 

17 

NON-PREC  APPC  H 

13 

16 

NVG  LDG 

14 

15 

NVG  LZ 

15 

14 

THREAT  RESPONSE 

16 

13 

RNAV  APPCH 

17 

12 

CIRCLE  APPCH 

18 

11 

HVY  FF  LDNG 

19 

10 

DAY  LOW  LEVEL 

20 

9 

NDB  APPCH 

21 

8 

HIGH  ALT  TAC 

22 

7 

LOWALTTAC 

23 

6 

CAT  II  APPCH 

24 

5 

MSN  CPU  APPCH 

25 

4 

LNDG  LZ  NIGHT 

26 

3 

HVY  FF  LDNG  NT 

27 

2 

NVG  LOW  LEVEL 

28 

1 

NVG  INST  APPC  H 

Table  A.l  SPR  Rank  vs.  Vol  1  Rank 


Appendix  B 


SKILL  REQUIRED  SCORE 

EVENT 

A 

B 

C 

D 

E 

F 

G 

H 

AVG 

RECEIVER  AR  AP  OFF 

26 

28 

27 

28 

27 

28 

24 

28 

27.00 

RECEIVER  AR  NIGHT 

27 

27 

28 

27 

28 

27 

25 

27 

27.00 

NVG  LZ 

25 

26 

25 

25 

24 

25 

27 

24 

25.13 

RECEIVER  AR 

28 

24 

26 

26 

26 

26 

15 

25 

24.50 

LNDG  LZ  NIGHT 

24 

25 

23 

24 

25 

14 

28 

26 

23.63 

LANDING  LZ 

23 

22 

24 

23 

23 

24 

21 

23 

22.88 

HVY  FF  LDNG  NT 

22 

12 

20 

22 

16 

19 

26 

22 

19.88 

HIGH  ALT  TAC 

21 

20 

14 

15 

19 

20 

17 

18 

18.00 

HVY  FF  LDNG 

19 

9 

19 

20 

15 

18 

18 

21 

17.38 

NVG  LDG 

15 

10 

18 

21 

13 

23 

20 

19 

17.38 

LOW  ALT  TAC 

20 

21 

15 

14 

18 

13 

16 

17 

16.75 

NVG  LOW  LEVEL 

18 

16 

17 

1 

21 

17 

23 

15 

16.00 

DAY  LOW  LEVEL 

10 

23 

21 

3 

22 

22 

12 

13 

15.75 

CIRCLE  APPCH 

6 

17 

22 

13 

17 

15 

6 

12 

13.50 

NT  LANDING 

13 

11 

6 

19 

6 

8 

22 

20 

13.13 

NVG  TAKEOFF 

14 

7 

8 

16 

12 

12 

19 

6 

11.75 

NDB  APPCH 

1 

19 

16 

12 

10 

16 

7 

11 

11.50 

TAC  DEPT 

11 

15 

9 

5 

14 

4 

9 

16 

10.38 

NVG  INST  APPCH 

16 

6 

5 

17 

11 

3 

14 

9 

10.13 

THREAT  RESPONSE 

7 

14 

7 

2 

7 

21 

8 

14 

10.00 

CAT  II  APPCH 

17 

2 

11 

11 

20 

9 

3 

4 

9.63 

LANDING 

12 

5 

4 

18 

5 

7 

13 

5 

8.63 

MSN  CPU  APPCH 

3 

18 

10 

9 

9 

2 

5 

8 

8.00 

NON-PREC  APPCH 

9 

8 

12 

8 

2 

11 

4 

7 

7.63 

RNAV  APPCH 

2 

13 

13 

10 

8 

1 

2 

10 

7.38 

INST.  APPCH 

5 

4 

2 

6 

4 

5 

10 

2 

4.75 

PRECISION  APPCH 

8 

1 

3 

7 

1 

10 

1 

3 

4.25 

TAKEOFF 

4 

3 

1 

4 

3 

6 

11 

1 

4.13 

Table  B.l  Skill  Required  Score  by  Sub-Category  and  Interviewee  (A-H) 


SKILL  DIMINISH  RATE  SCORE 

EVENT 

A 

B 

C 

D 

E 

F 

G 

H 

AVG 

RECEIVER  AR  AP  OFF 

28 

28 

27 

28 

27 

28 

26 

28 

27.50 

RECEIVER  AR  NIGHT 

27 

27 

28 

27 

28 

27 

25 

27 

27.00 

RECEIVER  AR 

26 

24 

26 

26 

26 

26 

24 

25 

25.38 

NVG  LZ 

24 

26 

25 

25 

25 

25 

27 

24 

25.13 

LNDG  LZ  NIGHT 

21 

25 

23 

24 

24 

17 

28 

26 

23.50 

LANDING  LZ 

23 

22 

24 

23 

22 

24 

21 

23 

22.75 

HVY  FF  LDNG  NT 

13 

12 

20 

22 

19 

16 

26 

22 

18.75 

NVG  LDG 

14 

10 

18 

21 

17 

22 

20 

19 

17.63 

DAY  LOW  LEVEL 

22 

23 

21 

3 

21 

21 

12 

13 

17.00 

HIGH  ALT  TAC 

19 

20 

14 

15 

13 

19 

17 

18 

16.88 

HVY  FF  LDNG 

12 

9 

19 

20 

18 

15 

18 

21 

16.50 

LOW  ALT  TAC 

18 

21 

15 

14 

12 

18 

16 

17 

16.38 

NVG  LOW  LEVEL 

25 

16 

17 

1 

20 

13 

23 

15 

16.25 

NT  LANDING 

3 

11 

6 

19 

10 

23 

22 

20 

14.25 

CIRCLE  APPCH 

11 

17 

22 

13 

23 

6 

6 

8 

13.25 

NVG  TAKEOFF 

4 

7 

8 

16 

16 

14 

19 

12 

12.00 

THREAT  RESPONSE 

20 

14 

7 

2 

7 

20 

8 

14 

11.50 

TAC  DEPT 

16 

15 

9 

4 

11 

11 

9 

16 

11.38 

NDB  APPCH 

9 

19 

16 

12 

8 

10 

7 

3 

10.50 

MSN  CPU  APPCH 

17 

18 

10 

9 

5 

5 

5 

9 

9.75 

NVG  INST  APPCH 

5 

6 

5 

17 

9 

12 

14 

10 

9.75 

LANDING 

10 

5 

4 

18 

15 

2 

13 

11 

9.75 

NON-PREC  APPCH 

15 

8 

12 

8 

2 

9 

4 

6 

8.00 

RNAV  APPCH 

7 

13 

13 

10 

3 

7 

2 

7 

7.75 

CAT  II  APPCH 

6 

2 

11 

11 

6 

8 

3 

5 

6.50 

TAKEOFF 

1 

3 

1 

5 

14 

1 

11 

2 

4.75 

INST.  APPCH 

2 

4 

2 

6 

4 

3 

10 

4 

4.38 

PRECISION  APPCH 

8 

1 

3 

7 

1 

4 

1 

1 

3.25 

Table  B.l  Skill  Diminish  Score  by  Sub-Category  and  Interviewee  (A-H) 


PROBABILITY  SCORE 

EVENT 

A 

B 

C 

D 

E 

F 

G 

H 

AVG 

TAKEOFF 

28 

28 

28 

27 

28 

28 

26 

28 

27.63 

LANDING 

27 

27 

27 

28 

27 

27 

27 

27 

27.13 

PRECISION  APPCH 

25 

25 

25 

21 

25 

26 

28 

26 

25.13 

INST.  APPCH 

26 

26 

26 

22 

26 

24 

25 

24 

24.88 

NON-PREC  APPCH 

19 

24 

23 

20 

24 

23 

24 

25 

22.75 

NT  LANDING 

24 

23 

24 

26 

22 

25 

5 

22 

21.38 

LOW  ALT  TAC 

22 

15 

22 

13 

20 

15 

13 

18 

17.25 

RNAV  APPCH 

10 

11 

5 

17 

23 

22 

23 

23 

16.75 

CAT  II  APPCH 

12 

14 

12 

18 

14 

21 

22 

19 

16.50 

NVG  LDG 

23 

20 

20 

24 

17 

8 

6 

13 

16.38 

HVY  FF  LDNG 

15 

12 

19 

25 

16 

17 

19 

6 

16.13 

NVG  TAKEOFF 

17 

19 

21 

23 

18 

9 

7 

14 

16.00 

HIGH  ALT  TAC 

21 

21 

4 

16 

21 

16 

12 

15 

15.75 

TAC  DEPT 

18 

22 

15 

19 

19 

5 

18 

10 

15.75 

CIRCLE  APPCH 

11 

8 

6 

15 

13 

19 

21 

20 

14.13 

HVY  FF  LDNG  NT 

14 

13 

18 

14 

15 

13 

20 

5 

14.00 

THREAT  RESPONSE 

20 

7 

16 

8 

7 

3 

11 

16 

11.00 

MSN  CPU  APPCH 

13 

9 

3 

1 

8 

18 

14 

21 

10.88 

RECEIVER  AR 

3 

16 

14 

11 

11 

11 

16 

4 

10.75 

NVG  INST  APPCH 

16 

17 

9 

12 

12 

2 

3 

9 

10.00 

LANDING  LZ 

6 

3 

17 

4 

6 

12 

17 

8 

9.13 

RECEIVER  AR  NIGHT 

4 

18 

8 

10 

10 

10 

8 

2 

8.75 

LNDG  LZ  NIGHT 

9 

4 

11 

5 

4 

14 

15 

7 

8.63 

NVG  LZ 

5 

5 

10 

6 

5 

6 

4 

11 

6.50 

RECEIVER  AR  AP  OFF 

2 

6 

13 

9 

9 

1 

9 

3 

6.50 

DAY  LOW  LEVEL 

7 

1 

2 

7 

1 

4 

10 

17 

6.13 

Table  B.l  Probability  Score  by  Sub-Category  and  Interviewee  (A-H) 


RISK  SCORE 

EVENT 

A 

B 

C 

D 

E 

F 

G 

H 

AVG 

RECEIVER  AR  AP  OFF 

26 

24 

23 

24 

27 

28 

27 

27 

25.75 

RECEIVER  AR  NIGHT 

27 

23 

24 

23 

28 

27 

26 

28 

25.75 

NVG  LZ 

25 

26 

21 

28 

24 

23 

25 

22 

24.25 

RECEIVER  AR 

28 

22 

22 

22 

26 

26 

20 

26 

24.00 

LNDG  LZ  NIGHT 

24 

25 

20 

26 

25 

19 

28 

21 

23.50 

LANDING  LZ 

23 

21 

19 

27 

23 

22 

19 

20 

21.75 

HVY  FF  LDNG  NT 

21 

16 

26 

20 

22 

18 

23 

24 

21.25 

DAY  LOW  LEVEL 

13 

27 

28 

25 

14 

24 

17 

18 

20.75 

NVG  LDG 

22 

14 

16 

21 

19 

21 

22 

14 

18.63 

NVG  LOW  LEVEL 

11 

20 

18 

14 

15 

25 

24 

19 

18.25 

HVY  FF  LDNG 

20 

13 

25 

15 

21 

17 

8 

23 

17.75 

THREAT  RESPONSE 

7 

28 

27 

12 

1 

12 

16 

25 

16.00 

NT  LANDING 

19 

15 

15 

19 

20 

5 

18 

15 

15.75 

NVG  TAKEOFF 

17 

7 

5 

16 

17 

15 

21 

13 

13.88 

LANDING 

18 

10 

14 

18 

18 

2 

13 

9 

12.75 

LOW  ALT  TAC 

3 

19 

17 

13 

10 

10 

10 

17 

12.38 

TAKEOFF 

16 

6 

9 

17 

16 

1 

14 

10 

11.13 

NVG  INST  APPCH 

10 

9 

7 

9 

8 

16 

12 

11 

10.25 

TAC  DEPT 

1 

17 

6 

11 

13 

3 

15 

16 

10.25 

CAT  II  APPCH 

15 

11 

13 

7 

11 

9 

4 

8 

9.75 

HIGH  ALT  TAC 

2 

18 

1 

8 

9 

11 

11 

12 

9.00 

CIRCLE  APPCH 

4 

12 

10 

10 

12 

7 

6 

7 

8.50 

NDB  APPCH 

8 

8 

8 

6 

7 

14 

3 

1 

6.88 

RNAV  APPCH 

6 

4 

11 

5 

6 

8 

5 

6 

6.38 

MSN  CPU  APPCH 

5 

5 

12 

4 

5 

6 

7 

5 

6.13 

PRECISION  APPCH 

14 

2 

2 

1 

3 

20 

1 

2 

5.63 

Table  B.l  Skill  Required  Score  by  Sub-Category  and  Interviewee  (A-H) 
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